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FOREWORD 



April 6,2011 



Dear Reader, 

The Portland Schools Foundation (PSF) is pleased to share this initial report on the Ninth Grade 
Counts (NGC) initiative that shows promising results for our youth. Four years ago, PSF published its 
Connected by 25 research that for the first time shined a light on Portland’s troubling graduation 
rates. This same research pointed to the ninth grade transition as a critical point in determining a 
student’s future success in school. In response to this data, schools, non-profit organizations, and the 
community mobilized to make a difference during the summer between the eighth and ninth grades. 
This report portrays some of the results of the collaborative work that developed as a result of Ninth 
Grade Counts. 

The data presented in these pages is appropriately timed as Portland and Multnomah County are 
currently embarking on an unprecedented effort to boost student success from "cradle to career" 
through improved collaboration and data-driven decision making. Our community has joined with 
four other cities nationally working to adapt the approach of the Cincinnati-based Strive partnership 
to build a lasting civic infrastructure focused on the success of every child, every step of the way, 
from cradle to career. Through this effort we will clearly measure both our challenges and the results 
of our efforts to address those challenges, allowing us to build a sense of shared accountability and a 
culture of continuous improvement. And so, as a community we can share in the good news 
presented in these pages: first, that our collective Ninth Grade Counts effort is reaching the students 
we’ve targeted — students of color, students in poverty, and students at high risk of dropping out — 
and second, the support we’re providing is having a positive impact on academic outcomes, 
particularly in the area of high school credit accumulation. Equally important as these outcomes is 
the fact that a diverse set of partners have come together, not only to take coordinated action, but 
also to focus those actions around research and share data on the results of our work in the interest 
of improving our efforts in the future. Through significant support from the City of Portland, the 
collaborative success of Ninth Grade Counts has inspired a series of supports for "academic priority” 
students, known as "the Summer Youth Connect” continuum, that provides key resources and 
opportunities to high-need students through high school and beyond. 

Of course academic data alone does not capture all of the complex factors that contribute to student 
success. In addition to the analysis presented in this report, PSF has gathered extensive survey data 
from students measuring Ninth Grade Counts’ impacts on behaviors, attitudes, and beliefs. This 
analysis reveals statistically significant growth over the course of the summer initiative on a vast 
majority of the "developmental assets” measured by the surveys. 1 



1 Portland State University Center for Student Success. 2010 NGC Student and Partner Surveys: Data Displays, Reporting and 
Analysis. 
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Highlights from the surveys include: 

•/ 95% of students reported that they were better prepared for ninth grade 
% 94% of students reported that they were more motivated to complete high school 
% 93% of students reported that NGC helped them to believe college is possible for them 

Working in partnership with the Office of the Mayor, the Office of the County Chair, Oregon Campus 
Compact, Marylhurst University, and others, the Portland Schools Foundation has been able to 
leverage significant annual in-kind support for our non-profit and school-sponsored partner 
programs — support that adds up to real impacts on today’s students. In fact, surveys of Ninth Grade 
Counts partner programs indicate that 93% to 100% of them believe that participating in the Ninth 
Grade Counts partnership increases their capacity, enhances their services, and helps them reach 
more students. 

The PSF staff and board would like to express our deep appreciation to the Northwest Evaluation 
Association whose generosity and expertise have made this report possible. And finally, Ninth Grade 
Counts would have been little more than an idea without contributions from the following funders 
who had the vision to invest in our students and our future: the City of Portland, Paul G. Allen Family 
Foundation, Portland General Electric, Qwest Foundation, and U.S. Bank. 

The challenges we face are very real, yet the community has demonstrated its ability to mobilize and 
act strategically in the interest of all students, using data as our anchor. Ninth Grade Counts is one 
compelling example, and as we move toward increased alignment through the Cradle to Career 
partnership, initiatives such as this can serve as a rallying cry and a proof point for the power of our 
collective impact. 

Sincerely, 

Dan Ryan 
CEO 

Portland Schools Foundation 
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EXECUTIVE SUMMARY 

The Portland Schools Foundation's (PSF) Ninth Grade Counts initiative is a network of more than twenty independent 
summer transition programs targeting Academic Priority (or "at-risk") students. These programs share a common focus 
on providing academic support, enrichment, and career/college exposure for students who show early warning signs for 
dropping out of school. The programs, which are sponsored by school districts, nonprofits, and colleges, vary in their 
format and curricula and are available to students identified as Academic Priority, as well as others, across Multnomah 
County (6 school districts) during the summer between eighth and ninth grades. The purpose of the Ninth Grade Counts 
summer transition programs is to re-engage Academic Priority students with their education through building skills, 
attitudes, and beliefs that lead to school success. The hope is that participation in one or more of the summer transition 
programs will be associated with increased likelihood of ninth grade enrollment, fewer absences during ninth grade, 
fewer failing grades in core academic content areas, fewer suspensions/expulsions, and greater numbers of completed 
academic credits at the end of the ninth grade school year. 



PARTICIPATION 

One section of this report details the sample of students in the study: Summer 2009 program participants and eighth 
grade Academic Priority students in 2008-2009 for whom data was available. 



Sample of students evaluated in Cohort 1: Eighth graders in 2008-2009 




• Ninth Grade Counts Academic Priority students were more likely to be in the free/reduced lunch program (a 
measure of family poverty) than other Academic Priority students but were not more likely to be English 
Language Learners or students of a certain gender. 

• Ninth Grade Counts served a larger percentage of students of color than was represented in the Academic 
Priority population or the total population of eighth graders. The "overrepresentation" of students of color in 
summer programs could be due to intentional partnerships with culturally-specific community organizations. 

• 43% of the 6663 eighth grade students in the six school districts were identified as Academic Priority in 08-09. 

• In Summer 2009, Ninth Grade Counts served 11% of eighth graders and 14% of Academic Priority eighth graders. 

• Of the six local school districts, Portland and Parkrose had larger percentages of Ninth Grade Counts participants 
than total students; David Douglas and Reynolds had the same percentage of students in the program as in the 
population at large; and Centennial and Gresham-Barlow had smaller percentages of NGC participants than total 
students, although none of their participants were Academic Priority students. 
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KEY FINDINGS 

The primary purpose of this report is to determine whether Academic Priority students who participated in a Ninth 
Grade Counts program during the summer of 2009 had better academic outcomes in the ninth grade than Academic 
Priority students who did not attend a program. 

• Ninth Grade Counts has a clear, positive effect on high school credit attainment. 

Academic Priority students who have participated in a Ninth Grade Counts program accumulated more high 
school credits by the end of ninth grade on average than nonparticipants, even when adjusted for possible 
credits received during the summer and even when controlling for potential selection bias. 

Credit attainment with and without adjustment for possible summer credits 





AP 

participant 


AP 

nonparticipant 


Number of Students 


392 


2313 






Total Number of Credits 


5.59* 


5.03* 


%of students with 6 or 
more credits 


56% 


49% 






Adjusted Number of 
Credits 


5.32 


5.03 


%of students with 6 or 
more adjusted credits 


51% 


49.0% 



Blue highlighting shows where the differences were statistically significant (p<.05) 

The asterisk indicates where differences were statistically meaningful (effect size >.03) 

• Other results were less clear. 

Academic Priority Ninth Grade Counts participants had higher attendance, lower Grade Point Average, and more 
suspensions than non-participants, but none of these results was statistically significant. Analysis of test scores 
and dropout rates was not possible with the data available. 

• There may be an effect for individual programs. 

Preliminary analysis suggests that some individual programs could show better outcomes for students, but more 
data is needed to analyze further. Very few programs had enough Academic Priority students to measure 
individual program differences between Ninth Grade Counts participants and nonparticipants, however the two 
large programs that were able to measured showed increased outcomes for participants. 

RECOMMENDATIONS 

In the process of performing this analysis of the Ninth Grade Counts summer 2009 program, some issues were identified 
that could increase the utility of future reports for this population. 

• Ensure all schools report grade point average statistics for ninth grade 

• Measure which students received high school credit for summer program participation 

• Collect the names of courses students have taken, along with grades 

• Perform a multi-year study involving a full high school cohort 
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INTRODUCTION 



The Portland Schools Foundation's (PSF) Ninth Grade Counts initiative is a network of more than twenty independent 
summer transition programs targeting Academic Priority (or "at-risk") students. These programs share a common focus 
on providing academic support, enrichment, and career/college exposure for students who show early warning signs for 
dropping out of school. The programs, which are sponsored by school districts, nonprofits, and colleges, vary in their 
format and curricula and are available to students identified as Academic Priority, as well as others, across Multnomah 
County (6 school districts) during the summer between eighth and ninth grades. The alignment of programs under the 
name of Ninth Grade Counts (NGC) was initiated in 2008, with the first round of quantitative data collected for students 
who participated in the summer of 2009. The six local participating school districts agreed to scan their entire eighth 
grade student body using a consistent set of early warning indicators (including grades, attendance, and test scores) to 
develop a list of Academic Priority students. These lists were used to help target students for participation in Ninth 
Grade Counts, as well as other supports, and also to serve as the basis for program evaluation. 

The purpose of the Ninth Grade Counts summer transition programs is to re-engage Academic Priority students with 
their education through building skills, attitudes, and beliefs that lead to school success. The hope is that participation in 
one or more of the summer transition programs will be associated with an increased likelihood of ninth grade 
enrollment, fewer absences during ninth grade, fewer failing grades in core academic content areas, fewer 
suspensions/expulsions, and a greater number of completed academic credits at the end of the ninth grade school year, 
all of which are indicators of future high school graduation. 

During the 2008-2009 school year, the Portland Schools Foundation consulted with research staff from Northwest 
Evaluation Association (NWEA), a local nonprofit educational services organization that works with school systems 
across the US with the mission to Partner to Help All Kids Learn. Together with representatives from the City of Portland 
Mayor's Office and the Education Cabinet, NWEA proposed an evaluation plan. In January 2010, research staff from 
NWEA provided the Portland Schools Foundation with a report displaying some of the early data available, mostly about 
student participation in summer 2009. PSF was provided with data to inform recruitment strategies for the 2010 Ninth 
Grade Counts students, and also to make adjustments needed in the data format for the June 2010 data collection from 
school districts. 

The data NWEA analyzed for this project included three categories of students from the cohort transitioning from eighth 
grade to ninth grade in summer 2009 (Cohort 1): eighth grade Academic Priority students who attended a Ninth Grade 
Counts program in summer 2009, eighth grade Academic Priority students who did not attend a Ninth Grade Counts 
program in summer 2009, and eighth grade students who attended a Ninth Grade Counts program in summer 2009 but 
weren't Academic Priority. Those populations are summarized in the figure below. 
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Figure 1: Number of students evaluated in Cohort 1 




The majority of the analyses in this report concentrate on the comparison between Academic Priority students who 
attended a program in summer 2009 (dark blue in Figure 1) and Academic Priority students who didn't attend a program 
(light blue in Figure 1). Ninth Grade Counts participants who were not Academic Priority (light green in Figure 1) were 
not included in most analyses because they were determined to be different enough from the Academic Priority 
students to be measured differently. Flowever general statistics about all three groups have been included, and a full 
table of data for all groups can be found in Appendix B. 

In addition, participation data was available for Cohort 2— students who participated in a summer 2010 Ninth Grade 
Counts program and Academic Priority students who were in the eighth grade in 2009-2010. Cohort 2 students are 
similar to Cohort 1 but are enrolled in the following school year. Cohort 2 students were not included in the majority of 
analyses since outcome data will not be available for them until October 2011. Flowever, Ninth Grade Counts program 
partners may want to use this information to help inform recruitment of participants for summer 2011 and beyond. 
Ninth Grade Counts partners will receive individual reports of demographic variables for both cohorts as well as average 
outcome variables for Cohort 1 for students in their respective programs. 
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METHODOLOGY 



DATA COLLECTION 

The Portland Schools Foundation along with the City of Portland, Multnomah County, and Worksystems, Inc. developed 
a Memorandum of Understanding (MOU) with each of the six participating school districts to collect confidential, 
student-level data for the purposes of this project. Districts agreed to submit anonymous data for all students who were 
designated as Academic Priority, as well as for all identified Ninth Grade Counts participants. Data collection was 
coordinated by the Multnomah Education School District (MESD) to make data collection more efficient for districts. 
MESD created customized data reports for each district using the ESIS student data management system identification 
numbers for each student, and created proxy identification numbers so students could be tracked confidentially over 
time. MESD released the files to each district, which in turn released the files to the Portland Schools Foundation and 
Northwest Evaluation Association (NWEA) for analysis. The resulting dataset included two years of school data for all 
identified Ninth Grade Counts participants during the summer of 2009 as well as all Academic Priority students in the six 
districts who were in the eighth grade in 2008-2009. This data is referred to as Cohort 1. The dataset also included one 
year of data for summer 2010 Ninth Grade Counts students and Academic Priority students in the eighth grade in 2009- 
2010 for preliminary analysis of Cohort 2. Outcome data for Cohort 2 will not be available until October 2011. The data 
element list provided by districts is included in Appendix C. 

Demographic Variables Measured 

An Academic Priority (AP) status was designated for each student in each year of data. Academic Priority students are 
designated as Academic Priority when they meet one or more of the following criteria: 

• Low or very low Benchmark scores on the Oregon Assessment of Knowledge and Skills (OAKS) in at least two 
subjects during eighth grade (Reading, Math, Science) 

• Received one or more non-passing grades during eighth grade in any core subject classes 

• Had 16 or more absences during the eighth grade school year 

Academic Priority is defined at the eighth to ninth grade transition, and typically once a student is identified as academic 
priority that designation remains until the student exits the system. For the purpose of this study, if a student was 
designated in either year, they were categorized as AP for both years. 

The following demographic elements were also provided. 

• School and district at which student completed eighth grade 

• School and district at which student completed ninth grade 

• Gender 

• Free/Reduced Price Lunch Eligibility - the most common indicator of family income available to schools. 

• Academic Priority status 

• English Language Learner status. Students categorized as English Language Learners are not yet fully proficient in 
English. 

• Ethnicity (Hispanic or not Hispanic) 

• Race (five categories) 
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For consistency, if a student was designated Free/Reduced Lunch Eligible in either year, they were categorized as FRL for 
both years. If a student was designated an English Language Learner or identified for ELL Monitoring in either year, they 
were categorized as ELL for both years. 

Although the ESIS data collection system has separate fields for ethnicity and race, not all districts use both fields. 
Because clear determinations could not be made for all districts, a new field was created called Race/Ethnicity. If a 
student was designated as Hispanic in either the race or ethnicity field, they were categorized as Hispanic. If a student 
was not designated as Hispanic, the race and/or ethnicity field was used for categorization. In cases where race or 
ethnicity was different between the two years of data, the more specific designation was used (instead of "other" or 
blank designations). For a few cases where race was different between the two years and wasn't "other" or blank in 
either case, the 2008-2009 designation was used. Missing or "multi" race designations were categorized as "other". 
"Asian" was categorized as "Asian/Pacific Islander" to conform to the standard. 

In addition to these demographic variables, the data file identified which Ninth Grade Counts programs, if any, each 
student attended. Some students attended more than one program. No students attended a program in more than one 
year. 

Outcome Variables Measured 



For each Ninth Grade Counts student or Academic Priority student identified, the data file included the following 
outcome variables for the student's eighth grade year: 

• Number of days present in school 

• Number of days absent from school 

• Number of expulsions 

• Number of out-of-school suspensions 

• Math testing data from the state assessment, including test score and performance level 

• Reading testing data from the state assessment, including test score and performance level 

• Cumulative Grade Point Average (GPA) as of the end of 2008-2009 

• Cumulative number of high school credits earned, if any, as of the end of 2008-2009 

• Number of high school credits earned and attempted for four core subjects (math, language arts, social studies, 
science) as of the end of 2008-2009. Total credits for these subjects may not equal total number of credits. 

For Cohort 1, the same data elements were also included for the 2009-2010 school year, in addition to the date each 
student entered ninth grade if applicable. Most students did not have high school credits at the end of the eighth grade, 
so only ninth grade cumulative credits were used. Most students did not take the state assessment tests in the ninth 
grade, so only eighth grade test scores were used. 

Four resulting datasets were used: 

• Unique eighth and ninth grade data for summer 2009 participants and Academic Priority students in the eighth 
grade in 2008-2009 

o Used for the majority of analyses: Cohort 1 
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• Unique eighth grade data for summer 2010 participants and Academic Priority students in the eighth grade in 
2009-2010 

o Used for student demographic counts for Cohort 2 

• eighth and ninth grade data for summer 2009 participants and Academic Priority students in the eighth grade in 
2008-2009, duplicated for every instance of a student in a Ninth Grade Counts program 

o Used for disaggregating analyses by individual program: Cohort 1 

• eighth grade data for summer 2010 participants and Academic Priority students in the eighth grade in 2009- 
2010, duplicated for every instance of a student in a Ninth Grade Counts program 

o Used for disaggregating student demographic counts by program for Cohort 2 

Each school within the six local school districts can input Grade Point Average data in the ESIS system or not, depending 
on preference. While most students have a valid GPA for the end of ninth grade, some students' GPA was listed as 0 
when in fact it was missing data. Because of this discrepancy in data collection, we removed the GPA for all students 
who were listed with 0.00 as their Grade Point Average. This may remove some students from analysis who truly had a 
0.00 GPA, but for most cases the reason for the 0.00 data was that the data was missing. Charter schools and alternative 
schools, for instance, do not report GPA and all of their students would show as 0.00 GPA. 317 out of 3185 students 
(10%) had 0.00 values removed. Another 169 (5%) had missing GPA values and were not included. 

For each student, data collection included the total number of accumulated high school credits, as well as the number of 
credits in four core subject areas. Some students received credits for participating in the summer NGC program, but 
these credits were not applied consistently by schools: some school districts awarded credit for NCG participation while 
others did not. In some cases within a single school district, some students received elective credit while others received 
core content elective credits; in other cases students may have only received credit if they completed the appropriate 
paperwork. When data were collected for this study, some NGC credits in one district were included in the count of core 
Language Arts or Social Studies credits, although subsequently these credits have been reclassified as elective credits not 
in core subject areas. However, in this case, if a student fails a core subject in the future, these credits may be 
reclassified again as core subject credits. Because NGC credits were handled in different ways for students, it was not 
possible to identify total credits without NGC credits included. Therefore, credits were aggregated in two standardized 
ways: total high school credits were calculated both as reported and also after subtracting credits that could have 
potentially been earned in the summer program; core content credits were calculated only as reported. More 
information about credit handling is included in the Findings section. 

DATA ANALYSIS 

Comparisons were made between participating and non-participating groups of Academic Priority students to determine 
whether involvement in one or more of the summer transition programs was associated with a measurable difference in 
any one or more of the outcome variables (for example, greater numbers of accumulated course credits, fewer failing 
grades, etc.). 

There are two main ways to measure whether differences between two groups of students are truly different. First, 
NWEA used a t-test to compare the difference between means on the outcome variables for the two groups (for 
example, comparing the mean number of accumulated course credits for Academic Priority participants in Ninth Grade 
Counts to the mean number of accumulated course credits for Academic Priority non-participants). A p value threshold 
of .05 was used to evaluate differences. Differences with p values under .05 are considered to be statistically significant, 

April 2011 12 | P a g e 




as they have a less than 5% probability of being chance differences. Another way of measuring the difference between 
two groups is to measure the average difference in means relative to the variance or standard deviation of the group. 
For instance, if the average difference in height for two groups of eight year old children was statistically significant but 
only .2 inches, it wouldn't represent a meaningful difference. Statistically, "effect size" as measured by Cohen's d 
indicates whether the difference in averages is a meaningful difference. It is calculated as the difference between means 
divided by the standard deviation of the study population. A Cohen's d value of .2 or .3 is considered a small effect; .5 a 
medium effect; .8 a large effect. 

Another way NWEA looked at outcome data was to measure the percentage of students achieving a certain benchmark 
level for each of the outcome measures. Sometimes the average score for a measure— for instance the average 
attendance rate— can be affected by extreme scores on either end. By measuring students against a common scale— for 
example, the percentage of students with a 90% attendance rate or higher— the comparison shows an additional layer 
of meaning. 

In order to minimize the potential for selection/recruitment bias, a statistical technique called "propensity score 
matching" was used to identify the most appropriate comparisons between the participating and non-participating 
groups. In this way, each participating student was compared to non-participating students who most closely resembled 
him or her. To determine which variables should be balanced to match NGC students with nonparticipants, NWEA ran a 
series of linear regressions to determine which factors predicted whether the student would enroll in a program and 
which factors predicted students' outcomes. 

For Academic Priority students, whether they enrolled in a program in summer 2009 is significantly correlated with the 
following variables from their eighth grade year: 

• District ID 

• School ID 

• Attendance rate 

• Math test score 

• Reading test score 

• Attendance passing (90%) rate 

• Math pass (Meets/Exceeds) rate 

• Read pass (Meets/Exceeds) rate 

• Free/reduced lunch status 

• Race/ethnicity 

• Belonging to a race/ethnicity category that is not white or Asian 

A linear regression model with all of those factors was determined to have a 7% influence on whether the student 
enrolled in a program (R-sq=.069). A series of regressions revealed that the best fit model for predicting whether a 
student enrolled in a program was a combination of district, Free/Reduced Lunch status, race/ethnicity, and eighth 
grade Reading score. This model predicted 6% (R-sq=.060) of selection bias, with all variables playing a significant role. 

Propensity Model 1: matching NGC students with nonparticipants based on four similar factors of district, 
free/reduced lunch status, race/ethnicity, eighth grade Reading score. 
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We also wanted to balance the dataset based on all available variables to see if results differed. 



Propensity Model 2: matching NGC students with nonparticipants based on all 10 available factors of 
free/reduced lunch status, English Language Learner status, gender, race/ethnicity, district, school, eighth grade 
Reading score, eighth grade Math score, eighth grade attendance, and eighth grade suspensions. 

NWEA ran a logistical regression on each of the propensity models to create a propensity score for each student. The 
propensity analysis was conducted in two ways for each of the models. First, each program participant was matched to a 
single nonparticipant with the closest propensity score. This limits the analysis to a one-to-one match. Secondly, NWEA 
matched each program participant with every nonparticipant who had a propensity score within the same quintile as the 
participant. In this way, all possible data was used to reduce the possibilities of a few outliers influencing outcomes. The 
statistically matched set of students was then analyzed to compare potential differences between Ninth Grade Counts 
Academic Priority participants and Academic Priority nonparticipants. 

LIMITATIONS 

The purpose of this report is to measure whether Academic Priority students in the summer 2009 cohort of Ninth Grade 
Counts had better outcomes in the year after participating in NGC than Academic Priority students who did not 
participate in NGC. This report is not a comprehensive evaluation of the effectiveness of the Ninth Grade Counts 
initiative. Intervention strategies often take several years for results to be seen, so a multi-year evaluation is the most 
appropriate methodology to reflect the result of overall program effectiveness. This report should be seen as a baseline 
study of program outcomes from which improvements can be designed and measured. 

In addition, this analysis represents a single year of data for the Ninth Grade Counts programs. Since one of the goals of 
the initiative is to increase the number of students graduating from high school, it would be important to track cohorts 
of eighth grade students participating in this program through high school to ascertain any possible impact the program 
might have on dropout and graduation rates. Lastly, Ninth Grade Counts is not a single program but a consortium of 
individual programs which may have varying levels of effectiveness. Since programs have very small groups of students 
(only six programs have more than 50 students), reliably measuring effectiveness of each individual program from one 
year's data is not realistic. Because the effectiveness associated with each program impacts the effectiveness of the 
collective group of programs, it is very important to come to some findings about the efficacy of the individual parts of 
the Ninth Grade Counts whole. As a track record is accumulated for some of the smaller programs over time, these kinds 
of analyses become more possible. 
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PARTICIPATION 



This section shows the number of students included in the analysis, separated by demographic characteristics, school 
district, and individual Ninth Grade Counts partner program name. It is important to note that not all students who 
participated are included in these tables— in 2009 (Cohort 1) participating programs reported to PSF that 830 students 
completed their respective programs; approximately 65 of those students completed more than one program. 
Approximately 60 students who attended programs (8%) could not be found in the student data management system, 
and thus are not included in this report. 



In the charts below and throughout the report, "Acad. Prior." or "AP" is used to denote Academic Priority students; 
"NGC" or "participant" is used to denote Ninth Grade Counts participants. The charts below represent Cohort 1 
students— students who participated in Ninth Grade Counts in the summer of 2009 plus Academic Priority students who 
were in the eighth grade in 2008-2009. 



Table 1: Proportion of students by demographic characteristics 



FRL 


N 


Y 


32% 


68% 


17% 


83% 


16% 


84% 


20% 


80% 



Gender 


F 


M 


44% 


56% 


48% 


52% 


44% 


56% 


52% 


48% 



ELL 


N 


Y 


77% 


23% 


75% 


25% 


75% 


25% 


75% 


25% 





Number of 
Students 


All Acad. Prior. 


2866 


All NGC participants 


718 


NGC Acad. Prior. 


399 


NGC non-Acad. Prior. 


319 



Table 2: Proportion of students by school district 





Number of 
Students 


All 8th graders in 6 districts 
in 2009-2010 


6663 


All Acad. Prior. 


2866 


All NGC participants 


718 


NGC Acad. Prior. 


399 


NGC non-Acad. Prior. 


319 



District 

_ ■ , David Gresham- „ , „ 

Centennial Parkrose Portland Reynolds Total 

Douglas Barlow 


8% 


12% 


15% 


4% 


49% 


12% 


100% 


11% 


13% 


13% 


5% 


42% 


15% 


100% 


2% 


9% 


7% 


16% 


55% 


11% 


100% 


0% 


11% 


0% 


15% 


55% 


19% 


100% 


4% 


8% 


15% 


17% 


55% 


1% 


100% 



Table 3: Proportion of students by race/ethnicity 



% of students in each 
race/ethnicity category 


Number of 
Students 


All 8th graders in 6 
districts in 2009-2010 


6663 


All Acad. Prior. 


2866 


All NGC participants 


718 


NGC Acad. Prior. 


399 


NGC non-Acad. Prior. 


319 



American 
Indian / 
Alaskan Native 


As i a n / 
Pacific 
Islander 


Black / 
African 
American 


Hispanic 


White 


Other 


Total 


1% 


10% 


11% 


18% 


55% 


4% 


100% 


2% 


7% 


16% 


23% 


51% 


<1% 


100% 


3% 


10% 


29% 


26% 


32% 


<1% 


100% 


5% 


7% 


28% 


26% 


34% 


<1% 


100% 


2% 


13% 


29% 


26% 


29% 


<1% 


100% 
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GENDER, RACIAL, and ECONOMIC REPRESENTATION 



• Ninth Grade Counts participants (both Academic Priority and non- Academic Priority) were more likely to be in 
the Free and Reduced Lunch program at their school than Academic Priority nonparticipants (NGC 83%, AP 68%). 
This data is used as a proxy for socioeconomic status, suggesting that more NGC participants came from low 
income families than nonparticipants. 

• The same approximate percentages of students were identified as English Language Learners (ELL) in all 
groupings of students: 23-25%. 

• Academic Priority students (both NGC participants and non-participants) had the same gender breakdown: 44% 
female and 56% male. Non-Academic Priority NGC participants had slightly more females participating: 52% 
compared to 48% male, making the overall NGC rate 48% female and 52% male. 

• Of the Academic Priority students, there were a higher percentage of students of color for Ninth Grade Counts 
participants than non-participants. This was particularly true for the African American students: 26% of AP 
participants were African American compared to 16% of the entire AP population. 

• Compared to the population of eighth graders in the six local districts, there was a higher percentage of Ninth 
Grade Counts participants who were Native American, African American, and Hispanic than the eighth grade 
population of these districts. The "overrepresentation" of students of color in the summer programs could be 
due to Ninth Grade Counts' intentional partnerships with cultural ly-specific community-based organizations. 

DISTRICT REPRESENTATION 

• Parkrose students were overrepresented compared to their population in the community: 16% of NGC 
participants were from Parkrose while only 4% of the total eighth grade population in the six districts was from 
Parkrose. This was driven mainly by a single Parkrose program which was the second largest of the NGC 
programs with 97 students. 

• There was a higher percentage of NGC participants from Portland Public Schools (PPS) compared to the 
percentage of students in the six districts: 55% of NGC participants were from PPS compared to 49% of total 
eighth grade students in the six districts coming from PPS. This may be due to the relative concentration of 
community-based organizations closer to the city core and/or the fact that PPS directly sponsored a relatively 
large program specifically for PPS students. 

• Centennial and Gresham-Barlow were both slightly underrepresented compared to the number of students in 
the area, and none of their participants were Academic Priority students. 

• David Douglas had approximately the same percentage of students in NGC as their representation in the 
community: 12%. The Reynolds School District also had a similar representation in the program as in the 
community (11-12%), however a higher percentage of their participants were Academic Priority students than 
the overall percentage of AP students in the six districts. 
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PROGRAM REPRESENTATION 



Table 4: Number of students by Ninth Grade Counts program 



NGC Program Name 


Summer 2009 
Students 


Students 
whose only 
NGC 

participation 
was in this 
program 


Big Brothers Big Sisters 


5 


4 


Campfire USA - Xploregon 


17 


12 


David Douglas School District - Ninth Grade Counts 


49 


45 


El Programa Hispano - Puentes 


57 


47 


1 Have A Dream Foundation 


15 


14 


1 RCO. -ASPIRE-SST 


19 


16 


IRCO - MCSI 


3 


3 


IRCO - Sabin SUN (plus Brothers & Sisters Keepers) 


25 


22 


1 RCO - SSSES SUN at Madison 


22 


20 


Metropolitan Family Service 


12 


5 


NAYA - Ninth Grade Summer Leaders 


14 


12 


Neighborhood House 


12 


12 


Open Meadow - Step Up 


184 


146 


Oregon Building Congress - Pre-ACE Academy 


2 


1 


Parkrose School District - Jumpstart - ELL Summer Program 


20 


12 


Parkrose School District - Summer Stampede Academy 


97 


87 


Portland Public Schools - 8th-9th Grade Summer Program 


75 


51 


REAP, Inc. - Challenge Camp 


20 


9 


Reynolds School District - N inth Grade Summer Transition 


62 


56 


Self-Enhancement, Inc. 


55 


55 


Straightway Services 


16 


13 


Urban League 


8 


8 



All instances of students in programs 


789 650 


Students in 2 programs 


65 


Students in 3 programs 


3 



All students 



718 



In addition to the Cohort 1 data shown, participation data was available for Cohort 2, students who participated in a 
summer 2010 Ninth Grade Counts program and Academic Priority students who were in the eighth grade in 2009-2010. 
These students were not included in the majority of analyses since outcome data will not be available for them until 
October 2011. However, Ninth Grade Counts partners may want to use this information to inform recruitment of 
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participants for summer 2011 and future years. Program staff will receive individual reports of demographic variables for 
both cohorts as well as average outcome variables for Cohort 1 for students in their program. 

Table 5: Student counts by subgroup for Cohort 1 (Summer 2009) and Cohort 2 (Summer 2010) 







Cohort 1: Summer 2009 




Cohort 2: Summer 2010 


Student subgroup 


All NGC 


NGC AP 


All AP 


6 districts* 




All NGC 


NGC AP 


All AP 


6 districts* 


Number of Students 




















Total 


718 


399 


2866 


6663 




745 


360 


2754 


6572 


Academic Priority 




















AP 


56% 


100% 


100% 






48% 


100% 


100% 




NotAP 


44% 










52% 








Race/Ethnicity 




















Native American 


3% 


5% 


2% 


1% 




2% 


2% 


2% 


1% 


Asian 


10% 


7% 


7% 


10% 




10% 


6% 


6% 


9% 


African American 


29% 


28% 


16% 


11% 




26% 


27% 


14% 


10% 


Hispanic 


26% 


26% 


23% 


18% 




28% 


28% 


22% 


21% 


White 


32% 


34% 


51% 


55% 




28% 


31% 


51% 


54% 


Other 


<1% 


<1% 


<1% 


4% 




6% 


6% 


5% 


5% 


Gender 




















M 


52% 


56% 


56% 


50% 




51% 


55% 


55% 


50% 


F 


48% 


44% 


44% 


50% 




49% 


45% 


45% 


50% 


Free or Reduced Lunch 




















FRL 


83% 


84% 


68% 


52% 




79% 


85% 


59% 


54% 


Not FRL 


17% 


16% 


32% 


48% 




21% 


15% 


41% 


46% 


English Language Learner 




















ELL 


25% 


25% 


23% 


12% 




24% 


26% 


20% 


12% 


Not ELL 


75% 


75% 


77% 


88% 




76% 


74% 


80% 


88% 


District 




















Centennial 


2% 


0% 


11% 


8% 




1% 


0% 


12% 


8% 


David Douglas 


9% 


11% 


13% 


12% 




10% 


7% 


12% 


12% 


Gresham-Barlow 


7% 


0 % 


13% 


15% 




10% 


18% 


14% 


14% 


Parkrose 


16% 


15 % 


5% 


4% 




11% 


10% 


5% 


4% 


Portland 


55% 


55% 


42% 


49% 




60% 


53% 


47% 


50% 


Reynolds 


11% 


19% 


15% 


12% 




9% 


13% 


9% 


12% 



* Data for the six districts from the Oregon Department of Education website, table # in parentheses. 

Number of students for eighth graders each year (#73) 
Race/ethnicity % for eighth graders each year (#67) 
Gender % from math assessments for eighth graders 2009-2010 only (#98) 

FRL % for all grades combined for each year (#61) 
ELL % from math assessments for eighth grade LEP students 2009-2010 only (#98) 

District % for eighth graders each year (#73) 
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FINDINGS 



SUMMARY STATISTICS 

There were a number of outcome variables that could be measured to compare the Ninth Grade Counts students who 
participated in summer 2009 programs to Academic Priority students in the same six school districts. The following 
tables show the overall mean (average) for each of the outcome variables in 2008-2009 (eighth grade) and 2009-2010 
(ninth grade). Note that high school credits and GPA are not measured for a valid number of students in eighth grade, 
and test scores are not measured for a valid number of students in ninth grade. 

Table 6: Average outcome measures by student type 



2008-2009 (8th grade) 


Student Type 


Number of 
Students 


GPA 


Expulsions 


Suspensions 


Attendance 

Rate 


Math Score 


Reading 

Score 


HS Credits 


All Acad. Prior. 


2866 




0.01 


0.43 


89.6% 


230.1 


229.7 




All NGC participants 


706 


0.00 


0.40 


92.7% 


229.7 


229.0 


NGC Acad. Prior. 


399 


0.00 


0.45 


91.0% 


227.4 


227.3 


NGC non-Acad. Prior. 


307 


0.01 


0.33 


94.9% 


232.7 


231.3 




2009-2010 (9th grade) 


Student Type 


Number of 
Students 


GPA 


Expulsions 


Suspensions 


Attendance 

Rate 


Math Score 


Reading 

Score 


HS Credits 


All Acad. Prior. 


2866 


1.95 


0.01 


0.37 


85.8% 






5.1 


All NGC participants 


706 


2.17 


0.01 


0.41 


89.4% 


6.1 


NGC Acad. Prior. 


399 


1.89 


0.01 


0.54 


86.9% 


5.6 


NGC non-Acad. Prior. 


307 


2.53 


0.01 


0.24 


92.6% 


6.8 



NWEA tested the statistical difference between three main groups of students: Academic Priority Ninth Grade Counts 
summer 2009 Participants, Academic Priority non-participants, and non-Academic Priority Ninth Grade Counts summer 
2009 Participants. The table below shows the averages for each group, and the light blue highlighting shows where the 
differences among the three groups were statistically significant (p<.05). 
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Table 7: Statistically significant differences in outcome variables 



AP 

participant 


AP 

nonparticipant 


Total 


2008_Expulsions 


0.00 


0.01 


0.01 


2008_Suspensions 


0.45 


0.42 


0.43 


2008_AttendanceRate 


91.0% 


89.4% 


89.6% 


2008_MathScore 


227.4 


230.5 


230.1 


2008_ReadingScore 


227.3 


230.1 


229.7 






2009_GPA 


1.89 


1.96 


1.95 


2009_Expulsions 


0.01 


0.01 


0.01 


2009_Suspensions 


0.54 


0.34 


0.37 


2009_AttendanceRate 


86.9% 


85.6% 


85.8% 


2009_HighSchoolCredits 


5.59 


5.03 


5.11 



As can be seen in the table above, the three groups had significantly different attendance rates and test scores in their 
eighth grade year before the summer program. AP participants had lower test scores than AP non-participants and non- 
AP participants, and higher attendance than non-participants but lower attendance than non-AP participants. 

During students' ninth grade year after the summer program, AP participants had more suspensions, driven in part by a 
few students with a large number of suspensions. Later in this report, the difference in number of students who had any 
suspensions will be addressed. After the summer program, AP participants had significantly higher number of high 
school credits than AP non-participants. 

Very few students had a valid high school credit or GPA data in the eighth grade, so credits and GPA before the program 
could not be measured. Only a small subset of students took the optional ninth grade assessment, so Math and Reading 
score change could not be measured. 

NINTH GRADE COUNTS PROGRAM IMPACT ON ACADEMIC PRIORITY STUDENTS FOR KEY OUTCOME MEASURES 

The main goal of the Ninth Grade Counts initiative is to keep students engaged in school to increase their likelihood of 
graduating from high school. Surveys collected from program participants suggested that students express being more 
engaged after participating in a program (see Foreword), but NWEA wanted to confirm if this shift in attitudes was 
visible in quantitative outcome data. 

Of the outcome variables that were available for this analysis, NWEA focused on three key variables to measure the 
difference between Academic Priority students who had participated in a Ninth Grade Counts program and Academic 
Priority students who hadn't participated: 

• Grade Point Average (GPA) 

• Attendance rate 

• High school credits accumulated 
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One of the clear measures for engagement in school is attendance rate, so NWEA analyzed attendance rates of 
participants and non-participants in the year before and the year after the summer program. Attendance decreased in 
aggregate between the two years, a predictable outcome for students as they transition from middle school to high 
school. NWEA also looked at the cumulative Grade Point Average (GPA) and the total number of high school credits 
earned by the end of their ninth grade year after the summer program. These two variables are important because 
successful completion of courses is one of the main predictors of graduation. 

Data was available for other outcome measures, but each had data collection issues which limited analysis. Only a 
fraction of students take the state test in the ninth grade, making comparison between the two school years invalid. 
Instances of suspensions and expulsion were in such small amounts (mostly 0 or 1 instances) that comparison of these, 
particularly change between two school years, is difficult to validate; in addition, discipline data such as this is often 
unreliable for comparison purposes due to inconsistencies in the way discipline is applied and recorded. 

The following section details the three main indicators of success— Grade Point Average, Attendance Rate, and High 
School Credits— and the differences between Academic Priority students who participated in a Ninth Grade Counts 
summer 2009 program and those that did not participate. 

Grade Point Average 

Because successful completion of courses is one of the main predictors of graduation, NWEA looked at cumulative grade 
point average at the end of the ninth grade. Grade Point Averages for Academic Priority students ranged from 0.05 to 
4.00 in 2009-2010, with a mean (average) of 1.95 and a median (midpoint) of 1.93. Students with a GPA of 0 were 
excluded from analysis because the 0 could have represented missing data. For more information about this exclusion, 
please see the Methodology section. Figures 2 and 3 below show ninth grade cumulative GPA for AP participants and AP 
nonparticipants. 



Figure 2: ninth grade GPA for AP NGC participants 



Figure 3: ninth grade GPA for AP nonparticipants 
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Figure 4: Academic Priority ninth grade Grade Point Average by level 



Grade Point Average 9th Grade 




A (3.50-4.00) B (2.50-3.49) C (1.50-2.49) D (1.00-1.49) F(.01-.99) 



Figure 4 shows that AP participants had a slightly lower percentage of students receiving A- or F-level grade point 
averages, and a slightly higher percentage of students receiving B-, C-, and D-level grade point averages. NWEA tested 
for statistical significance and found no real difference between AP participants and AP non-participants. There wasn't a 
significant difference between the percentage of students in each grade level (A-F) as shown above. There also wasn't a 
significant difference in the ninth grade numeric GPA for the two groups. 

Table 8: Grade Point Average benchmarks 



2009-2010 Data | 



Measure 


All NGC 


NGC AP 


Non- 

NGCAP 


All AP 


Number of Students 


660 


365 


2039 


2404 


Average GPA 


2.17 


1.89 


1.96 


1.95 


% of students with GPA over 1.50 


73% 


65% 


66% 


66% 


% of students with GPA over 2.00 


58% 


47% 


49% 


49% 


% of students with GPA over 2.50 


42% 


29% 


32% 


32% 



NWEA "benchmarked" grade point average by looking at the percentage of students whose GPA was 1.50 or higher, 

2.00 or higher, and 2.50 or higher in the school year after the summer program. There was not a statistically significant 
difference between participants and nonparticipants at each level. Note that students with a 0.00 GPA have been 
removed from this analysis because a large percentage of those students had missing data. See the Methodology section 
for more information. 
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Attendance Rate 



One of the clear measures for engagement in school is attendance rate, so NWEA analyzed attendance rates of 
participants and non-participants in the year before and the year after their participation in NGC. Attendance decreased 
in aggregate between the two years, a predictable outcome as students transition from middle school to high school. 

For Academic Priority students, attendance rates ranged from 27.5% to 100% with an average of 90% in the eighth grade 
and 86% in the ninth grade. Figures 5 and 6 below show ninth grade attendance rates; Table 9 shows eighth and ninth 
grade attendance rates. 

Figure 5: ninth grade attendance for NGC AP participants Figure 6: ninth grade attendance for AP nonparticipants 





9th grade attendance rate 9th grade attendance rate 

Table 9: Attendance rates for NGC participants and non-participants 



Measure 


AP participant 


AP 

nonparticipant 


Number of Students 


391 


2301 


2008-2009 Attendance Rate 


91.0% 


89.4% 


2009-2010 Attendance Rate 


86.9% 


85.6% 


Attendance Rate Change 


-3.94% 


-3.91% 



Only the attendance rate for eighth grade students before participating in NGC showed a statistically significant 
difference between Academic Priority NGC participants and non-participants: participants had higher attendance rates 
before the program than non-participants. Although attendance rates during the ninth grade year were higher for Ninth 
Grade Counts AP participants than AP nonparticipants, differences were not statistically significant, and the change 
between eighth and ninth grade attendance rates for the two groups was also not statistically significant. (Note that the 
Attendance Rate Change is each student's difference in eighth and ninth grade attendance rates, aggregated to the total 
for each group. It is not the difference between each group's average.) 
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Sometimes the mean (average) for a group of students hides differences between groups because particularly low or 
high cases distort the mean. For that reason, NWEA also analyzed the percentage of students who achieved a certain 
level of attendance. Using 80%, 85%, and 90% attendance rates as benchmark levels, NWEA found a similar result: the 
percentage of students with high attendance rates was better for NGC participants before the summer program. 

Table 10: Attendance rate differences for Academic Priority students 







AP participant 


AP 

nonparticipant 






398 


2460 










2008-2009 


90%or greater 


63 . 8 % 


55 . 0 % 


2009-2010 


90%or greater 


49 . 0 % 


49 . 8 % 










2008-2009 


85%or greater 


82 . 9 % 


77 . 2 % 


2009-2010 


85%or greater 


66 . 1 % 


65 . 5 % 










2008-2009 


80%or greater 


90 . 5 % 


87 . 9 % 


2009-2010 


80%or greater 


78 . 8 % 


75 . 4 % 



As can be seen in Table 10, the group of students who participated in a NGC program during the summer of 2009 had a 
larger percentage of students with high attendance in either school year and at any level of attendance. The only 
exception was the 2009-2010 school year at 90% attendance where both groups had similar rates. The light blue shading 
shows where the differences for the two groups were statistically significantly different. 

NWEA also limited the analysis to students who were enrolled in school for 50 or more days during the school year, in 
case the outlier students were affecting the results. NWEA found the same pattern when limiting the sample: 
participants had higher eighth grade attendance rates than nonparticipants in both years on most measurements. 

Table 11: Attendance rate differences for Academic Priority students with 50 or more school days 
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High school credits 



One of the main indicators of successful completion of high school is the number of high school credits accumulated 
during each academic year. Students who fall behind in credit attainment are more likely to drop out of high school. 
Although some credits can be made up in later years, credit accumulation, nonetheless, is one of the strongest 
predictors that a student will graduate. In fact, PSF's 2007 report, Connected by 25: The Fourth R (which informed and 
inspired the NGC initiative) 2 showed that a student with insufficient credits at the end of ninth grade has 4.1 times the 
risk of leaving school without graduating than a student with sufficient credits. The following section describes the 
findings for credit attainment between Academic Priority Ninth Grade Counts participants and Academic Priority 
students who did not participate. Figures 7 and 8 below show credits accumulated for AP participants and AP 
nonparticipants. 



Figure 7: High School Credits Earned 
for AP NGC participants 



Figure 8: High School Credits Earned 
for AP nonparticipants 




High School Credits Earned bv the end of 9th grade 




As can be seen in the charts above, the majority of Academic Priority students had between six and seven credits at the 
end of the ninth grade. However, because there were a large number of students with very few credits, the mean 
(average) number of credits accumulated was 5.11 for all students, 5.59 for participants, and 5.03 for nonparticipants. 
Very few students earned 10 or more credits. 

In addition to looking at the individual and average number of high school credits earned, NWEA also wanted to examine 
differences between AP participants and AP nonparticipants at different levels of credit attainment. For instance, are 
NGC students less likely to be in very low credit levels; or are NGC students more likely to be in the highest credit levels? 
NWEA found that for every level of credit attainment— more than two credits, more than eight credits, and each one- 
point credit in between— Ninth Grade Counts students averaged better performance. For instance, 89% of AP NGC 



2 http://www.connectedbv25.org/resources/ 
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participants earned more than two credits while only 84% of AP non-participants earned more than two credits. This 
was true for students with the most credits as well: 19% of AP NGC participants earned eight or more credits while only 
13% of non-participants did. All differences were statistically significant except the level of seven or more credits. The 
average number of credits attained was also statistically significantly higher for NGC participants than nonparticipants. 

Table 12: High School Credits Earned by the end of ninth grade 



High School Credits 


AP 

participant 


AP non- 
participant 


Statistically 

Significant 


Number of students 


392 


2313 


Y 


2 or more credits 


89.0% 


83.8% 


Y 


3 or more credits 


83.4% 


77.7% 


Y 


4 or more credits 


74.2% 


68.0% 


Y 


5 or more credits 


65.6% 


59.4% 


Y 


6 or more credits 


56.4% 


49.0% 


Y 


7 or more credits 


36.5% 


32.2% 


N 


8 or more credits 


19.4% 


13.0% 


Y 


Total HS Credits Earned 


5.59 


5.03 


Y 



One of the benefits of participating in Ninth Grade Counts is that some programs offer high school credits for completion 
of the summer program. During the summer of 2009 nine of the 22 programs offered .5 credits to students who 
completed the program, and two programs offered .25 credits. The following is a list of the programs that reported to 
PSF that they awarded credit to students participating in the 2009 Ninth Grade Counts program. All programs awarded 
.5 high school elective credits unless noted. 

• Campfire USA - Xploregon 

• David Douglas School District - Ninth Grade Counts 

• El Programa Hispano - Puentes 

• IRCO - ASPIRE-SST 

• IRCO -SSSES SUN at Madison 

• NAYA - Ninth Grade summer Leaders 

• Neighborhood House 

• Parkrose School District - Jumpstart - ELL summer Program 

• Portland Public Schools - eighth-ninth Grade summer Program 

• Self-Enhancement, Inc. (.25 credits) 

• Step Up (.25 credits) 

Not every student in one of these credit-granting programs received credits, however, because some students were 
ineligible due to district placement, completion, or other factors. Four of the ten programs estimated that only a fraction 
of their students actually received the credit. Overall, the combination of programs estimated that 80% of students in 
those programs received credit. For the purpose of this analysis, however, NWEA assumed that all students in a credit- 
granting program received the full credit possible in order to show a conservative estimate. Students who were in two 
credit-granting programs were assumed to have been given credit from only one program. The following table shows 
the percentage of students who were from credit-granting programs. 
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Table 13: Percentage of students in programs that awarded high school credit for the NGC summer program 



NGC 

credits 


AP 

participant 


AP 

nonparticipant 


Non-AP 

participant 


.00 


34% 


1 00% 


30% 


.25 


25% 


NA 


34% 


.50 


41% 


NA 


36% 


Total 


1 00% 


1 00% 


1 00% 



After subtracting out the full credit amount possible from every student who could have received it, NWEA performed 
the credit analysis again to see if the increase in credit attainment for NGC participants was solely due to the credit they 
received in the program. Among Academic Priority students, there was a significant difference (p=.014) between 
participants and nonparticipants in the total number of credits earned at the end of ninth grade, even when possible 
credit received from the summer program was subtracted for all students in credit-granting programs. 

Table 14: Credit attainment with and without adjustment for possible summer credits 





AP 

participant 


AP 

nonparticipant 


Number of Students 


392 


2313 






Total Number of Credits 


5.59* 


5.03* 


%of students with 6 or 
more credits 


56% 


49% 






Adjusted Number of 
Credits 


5.32 


5.03 


%of students with 6 or 
more adjusted credits 


51% 


49.0% 



As can be seen in Table 14, Ninth Grade Counts participants had a higher number of credits than nonparticipants, overall 
as well as when all possible credits earned in the program were subtracted. Blue highlighting shows where the 
differences were statistically significant (p<.05); an asterisk after the value indicates where differences were statistically 
meaningful (effect size >.03). 

In addition to looking at overall credit attainment, NWEA also looked at credit attainment in key subject areas. Table 15 
shows that NGC Academic Priority participants both attempted and earned more credits than AP nonparticipants in 
Language Arts and Science. Possible credits earned through participation in NGC were not subtracted from these 
numbers since credits earned were electives, not core subjects. See Methodology section for a more detailed 
explanation of how credits were assigned and collected. 
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Table 15: High school credits earned and attempted by Academic Priority students in core subject areas 





Earned 


Attempted 


"Pass" rate 


Participant Math 


0.68 


0.98 


70.0% 


Nonparticipant Math 


0.69 


0.99 


69.9% 



Participant Language Arts 



0.94 



1.30 



72.2% 



Nonparticipant Language Arts 

Participant Science 
Nonparticipant Science 



73.3% 




Participant Social Studies 


0.32 


0.48 


67.1% 


Nonparticipant Social Studies 


0.33 


0.48 


69.5% 



NGC participants attempted and earned statistically significantly more credits in Language Arts and Science than 
nonparticipants. There wasn't a significant difference between amount of Math and Social Studies credits earned or 
attempted between the two groups. There also was not a difference between the likelihood of students in different 
groups gaining one or more credits in each of the subjects, nor was there a difference in the "passing rate" defined as 
the number of credits earned divided by the number of credits attempted in each subject. The lack of statistical 
differences could be due to the small amount of credits accumulated in each subject by the end of ninth grade: most 
students had between zero and one-and-a-half credits in each subject. Reviewing credit accumulation at the end of 
tenth, eleventh, and twelfth grade might show more statistical difference. 

PROPENSITY SCORE ANALYSIS 

One challenge with measuring the impact of a program is when assignment to the program is not random. For instance, 
are students with high attendance more likely to sign up for the program? If so, does that explain why their attendance 
is higher after the program? Or are certain races or ethnicities more likely to sign up for a program because they 
received targeted outreach? Do the factors that influence a student to enroll in a Ninth Grade Counts partner program 
have an influence on their success after the program? 

One way to account for the chance that the students who signed up for Ninth Grade Counts were statistically different 
from students who didn't enroll, and that those differences have an effect on outcomes, is to conduct a propensity score 
analysis. Propensity score analysis is often conducted in health trials where assignment to a group is not random; for 
instance when testing a diet program where people were given a choice as to whether or not to sign up. Propensity 
score analysis matches each "treatment" participant— here that means Academic Priority Ninth Grade Counts student— 
with an AP nonparticipant who is statistically most similar. By matching participants with non-participants on key 
characteristics, some of the difference on outcome variables that might be attributed to systematic selection bias may 
be mitigated. While this approach is very helpful, one cannot entirely mitigate the fact that participants who were 
motivated to participate in this program may be inherently more motivated toward school than those who chose not to 
participate. 
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To determine which variables should be balanced to match AP NGC students with AP nonparticipants, NWEA ran a series 
of linear regressions to determine which factors predicted whether the student would enroll in a program and which 
factors predicted students' outcomes. 

For Academic Priority students, whether they enrolled in a program in summer 2009 is significantly correlated with the 
following variables from their eighth grade year: 

• District ID 

• School ID 

• Attendance rate 

• Math test score 

• Reading test score 

• Attendance passing (90%) rate 

• Math pass (Meets/Exceeds) rate 

• Read pass (Meets/Exceeds) rate 

• Free/reduced lunch status 

• Race/ethnicity 

• Belonging to a race/ethnicity category that is not white or Asian 

A linear regression model with all of those factors was determined to have a 7% influence on whether the student 
enrolled in a program (R-sq=.069). A series of regressions revealed that the best fit model for predicting whether a 
student enrolled in a program was a combination of district, Free/Reduced Lunch status, race/ethnicity, and eighth 
grade Reading score. This model predicted 6% (R-sq=.060) of selection bias, with all variables playing a significant role. 

Propensity Model 1: matching NGC students with nonparticipants based on four similar factors of district, 
free/reduced lunch status, race/ethnicity, eighth grade reading score. 

We also wanted to balance the dataset based on all available variables to see if results differed. 

Propensity Model 2: matching NGC students with nonparticipants based on all 10 available factors of 
free/reduced lunch status, English Language Learner status, gender, race/ethnicity, district, school, eighth grade 
Reading score, eighth grade Math score, eighth grade attendance, and eighth grade suspensions. 

NWEA ran a logistical regression on each of the propensity models to create a propensity score for each student. The 
propensity analysis was conducted in two ways for each of the models. First, each program participant was matched to a 
single nonparticipant with the closest propensity score. This limits the analysis to a one-to-one match. Secondly, each 
program participant was matched with every nonparticipant who had a propensity score within the same quintile as the 
participant. In this way, all possible data was used to reduce the possibilities of a few outliers influencing outcomes. 
Figures 6 and 7 show the distribution of propensity scores, called "predicted probability." The figures show clear 
differences between the probability distributions associated with program participants and non-participants. This would 
indicate that the regression had some value for differentiating the two groups. The low values (most probabilities fall 
below 50%) with little variation between students confirms the results of the regression: the variables available in the 
analysis did little to predict whether a student would participate in a program or not. Some possible differences that 
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could explain selection bias include parental encouragement, differentially effective recruitment strategies, or 
differential degrees of risk among students. 

Figure 9: Propensity score for NGC participants Figure 10: Propensity score for non-participants 





Predicted probability Predicted probability 

The propensity scores were low overall and the regression results indicated that very little of the selection bias could be 
explained by available variables, however the distributions shown in Figures 9 and 10 indicate that participants had a 
measurably greater probability of being participants than nonparticipants using our model, so we continued with the 
propensity score analysis. The results of the first propensity score analysis (each participant matched to a single 
nonparticipant) showed that there was not a statistical difference between each Ninth Grade Counts student and the 
nonparticipant who most resembled them in ninth grade for the following outcome measures: ninth grade attendance 
rate, ninth grade suspension rate, ninth grade GPA, percentage of students with no suspensions, percentage of students 
with 90% or greater attendance, percentage of students with 1.50 GPA or higher. This was true when students were 
matched for the four variables that predicted participation in the program (district, free/reduced lunch status, 
race/ethnicity, and eighth grade Reading score) and also when students were matched for all ten available variables 
(district, school, free/reduced lunch status, English Language Learner status, gender, race/ethnicity, eighth grade 
Reading score, eighth grade Math score, eighth grade suspensions, and eighth grade attendance). 

The difference in the number of credits accumulated by the end of ninth grade was significantly higher for students who 
participated in Ninth Grade Counts, even when potential bias was eliminated. NGC participants also were more likely to 
have six credits or more at the end of ninth grade than nonparticipants when selection bias was controlled. In addition, 
NGC participants accumulated more credits than nonparticipants even when the full amount of potential summer credit 
was subtracted for every student, regardless of whether they actually received credit for participating. The number of 
adjusted credits was not statistically significant between participants and non-participants, but this may be due to the 
fact that correcting for potential summer credits underestimated actual credit accumulation for some participants. For 
more information about this issue, please see the Findings section which addresses credit accumulation, as well as the 
Recommendation section which addresses data collection of summer participation credits. 
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For the second propensity analysis, NWEA divided the population into quintiles of propensity score, roughly translating 
to how likely the student was to have enrolled in one of the NGC partner programs. A weighted average of adjusted high 
school credits (high school credits at the end of ninth grade minus full possible credits for program participants) was 
created from each of these quintiles for AP participants and AP nonparticipants. Although the sample sizes were too 
small to show statistical significance, the result showed that program participants averaged 5.26 weighted adjusted 
credits while nonparticipants averaged 5.07 weighted adjusted credits. This suggests that students who attend a Ninth 
Grade Counts partner program will gain more credits during the ninth grade than nonparticipants, even when the 
summer credits are excluded and even when selection bias is mitigated. 

Table 16: Adjusted high school credits by propensity quintile 



.V. n , „ . . Average Adjusted 

Number of students 

Credits 



Propensity 

Quntile 


AP 

Participant 


AP Non- 
participant 




AP 

Participant 


AP Non- 
participant 


1 


17 


531 


4.53 


5.31 


2 


42 


507 


5.10 


5.29 


3 


67 


481 


5.75 


4.85 


4 


100 


449 


5.34 


4.93 


5 


163 


385 


5.13 


4.88 


Total 


389 


2353 




Weighted 

average 




5.26 


5.07 



Average adjusted credits equals total high school credits for AP nonparticipants and AP participants in programs that didn't award summer credit; 
and equals total high school credits minus all possible credits earned for participation in the summer program for participants in programs that 
awarded any summer credit 



OTHER ANALYSES 

Dropout Data 

Data for this analysis was collected by the Multnomah Education Service District using the ESIS student information 
system. Since not every district in Oregon uses the ESIS system and ESIS cannot track students outside the state, 
students who were in the system in eighth grade and not in the system in ninth grade are not necessarily dropouts. 

Some may have transferred to other districts in the state, and some may have left the state. 

Dropout reporting is a complex endeavor involving multiple years of data. Dropouts are determined in the fall following 
a year of data. Students who are in the system in one year and are not found in the system in the next fall are potential 
dropouts. Oregon relies on the state-wide student information system which identifies students in any district within the 
state. If a student moves to a new district within Oregon, the statewide system informs the prior districts who can 
identify that student as a transfer as opposed to a dropout. For students who leave a district but cannot be found 
elsewhere in the state-wide system, districts are responsible for determining which of those students are drop outs and 
which have other reasons for not being in the system— for example if they have left the state or country, or if they have 
received a GED from another institution. The dropout information from the state system was not available for this 
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analysis, and even if it were to be included there would be challenges with using it in a single-year analysis such as this. 
For meaningful analysis, dropout information should be examined over the course of the student's entire high school 
career, not just within a single year. Some students who may not come to a school during the fall and would be counted 
as a dropout, may come back to school in the spring and future years to finish high school. Students who spend a 
semester studying abroad, students who return to their home country for a period of time, and students who spend part 
of the year with each of their parents in different states could all be inadvertently misidentified as dropouts if the school 
is not informed of these conditions. Conversely, there might be students who should be considered dropouts that 
wouldn't be under certain circumstances. For instance, if a district believes a student has left the state when they have 
not, or if the student left a school to get a GED but never completed it, or if a student leaves a school and goes to 
another one for at least one single day before dropping out, then these students may inadvertently be credited with 
staying in school when in fact they've left 

As noted in the Recommendations section of this report, NWEA recommends that a multi-year evaluation be conducted 
for Ninth Grade Counts that includes analysis of dropout data. 

With the data collected, only 17 students were in the ESIS system in the eighth grade but not in the ninth grade: one was 
an Academic Priority NGC participant, one was a non-Academic Priority NGC participant, and the other 15 were 
Academic Priority non-participants. Although these findings indicated that Ninth Grade Counts students could be less 
likely to drop out after eighth grade than nonparticipants, the data limitations prevent us from drawing a clear 
conclusion. 

Differences by Program 

One of the challenges in determining the effect of the Ninth Grade Counts initiative on ninth grade outcomes is that it is 
not a single program; the summer 2009 included 22 separate programs, while the summer of 2010 included 18 
programs. Most of the programs are small, which means that determining clear outcomes for those programs is difficult. 

Of the six programs with more than 50 students, only two— Reynolds and Step Up— had more than 50 Academic Priority 
students included in the available dataset. Both programs had results that exceeded comparable students on most 
indicators. In the Reynolds program, for instance, 65% of Academic Priority students passed the GPA benchmark of 1.50 
compared to 55% of Academic Priority students in the Reynolds School District, and 58% passed the 90% attendance 
benchmark compared to 47% in the district. In Step Up, as another example, the average cumulated credit attainment 
(after it was adjusted for possible summer credits) was 5.78 for Academic Priority participants from Portland or 
Gresham-Barlow compared to 5.07 for Academic Priority nonparticipants in those districts. 

The purpose of this report is to analyze the results of the combination of Ninth Grade Counts programs as a single 
initiative, not to report results of individual programs. Individual programs will receive a report of their participation and 
overall outcome data when sample sizes are large enough to protect student confidentiality. Appendix C provides an 
example of the individual reports created for each NGC partner. 
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Discipline Data 



The number of out-of-school suspensions and expulsions reported for each student in the dataset was low— 0 for 80% of 
students and 1 for 11% of students. The low amount per student makes comparison of this outcome variable difficult, 
particularly by using a "suspension rate" or "expulsion rate" measure. The following section provides information about 
disciplinary data for the Academic Priority population during the two-year time frame examined in this report. 

Of the 548 Academic Priority students who had a suspension or expulsion in the ninth grade, 110 had participated in 
Ninth Grade Counts. This means that 28% of Academic Priority NGC participants had some sort of disciplinary action 
during the ninth grade compared to 19% of Academic Priority nonparticipants. In the eighth grade, the rates of discipline 
were more consistent between the two groups: 23% of AP participants compared to 21% of AP nonparticipants. The 
pattern of number of suspensions per student was consistent between the two groups, as shown in Figures 11 and 12. 

Figure 11: ninth grade suspensions for NGC participants Figure 12: ninth grade suspensions for nonparticipants 





There was also an element of race/ethnicity observed in the discipline data. Of the 19 Academic Priority students who 
had an expulsion in eighth grade, 84% were students of color (African American, Hispanic, or Native American), and 
none of them enrolled in a Ninth Grade Counts program. In the ninth grade, 64% of the 36 students with an expulsion 
were students of color. Since 50% of Academic Priority students in the data are of color, these numbers indicate that 
students of color are disproportionally expelled from school compared to White students. No Asian students were 
expelled in either school year. While these numbers only reflect the Academic Priority population in Multnomah County, 
this overrepresentation appears in other areas of Oregon and nationally. This finding is not related to participation in the 
Ninth Grade Counts program, but is related to the Academic Priority population and may be an interesting area of 
further research. 
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DISCUSSION 



The purpose of this report is to determine whether Academic Priority students who participated in Ninth Grade Counts 
during the summer of 2009 had better academic outcomes in the school year following the summer between eighth and 
ninth grade than Academic Priority students who did not participate in NGC. The outcome variables available for the 
school year following the summer program included the following: 

■ Attendance rate 

■ Number of suspensions and expulsions 

■ Grade Point Average (GPA) 

■ Cumulative number of high school credits; credits earned and attempted in core subjects 

In addition to looking at average values for each of the outcome variables, NWEA also measured the percentage of 
students achieving a certain benchmark level for each of the outcome measures. The following table shows the average 
score for each of the measures, as well as the percentage of students who achieved the stated benchmark level of 
performance in each category. The table shows the school year before the NGC summer program (2008-2009), as well as 
the school year after the program (2009-2010). 

Table 17: Outcome averages and percent of students meeting "benchmarks" for Cohort 1 



2008-2009 Data ■ 2009-2010 Data I 



Measure 


All NGC 


NGC AP 


Non- 

NGCAP 


All AP 


Measure 


All NGC 


NGC AP 


Non- 

NGCAP 


All AP 


Numberof Students 


718 


399 


2467 


2866 




Numberof Students 


718 


399 


2467 


2866 


Attendance rate 


92.7% 


91.0% 


89.4% 


89.6% 




Attendance rate 


89.4% 


86.9% 


85.6% 


85.8% 


% of students with 
attendance >90% 


75% 


64% 


55% 


56% 




% of students with 
attendance >90% 


62% 


49% 


50% 


50% 


Suspension rate 


0.40 


0.45 


0.42 


0.43 




Suspension rate 


0.41 


0.54 


0.34 


0.37 


% of students without a 
suspension or expulsion 


80% 


77% 


79% 


79% 




% of students without a 
suspension or expulsion 


78% 


72% 


82% 


80% 


Math score 


230 


227 


231 


230 




GPA 


2.17 


1.89 


1.96 


1.95 


% of students meeting Math 
standard 


51% 


40% 


51% 


50% 




% of students with GPA over 
1.50 


73% 


65% 


66% 


66% 


Reading score 


229 


227 


230 


230 




High School Credits 


6.13 


5.59 


5.03 


5.11 


% of students meeting 
Reading standard 


47% 


38% 


49% 


48% 




% of students with 6 or more 
credits 


64% 


56% 


49% 


50% 














High School Credits Adjusted 


5.86 


5.32 


5.03 


5.07 














% of students with 6 or more 
credits adjusted 


59% 


51% 


49% 


49% 



In the year prior to the summer 2009 program, Ninth Grade Counts Academic Priority participants had significantly 
lower Reading scores, lower Math scores, and higher attendance rates compared to Academic Priority nonparticipants. 
Of the outcome variables that were available for this analysis, we focused on three key variables to measure the 
difference between Academic Priority students who had participated in a Ninth Grade Counts program and Academic 
Priority students who did not participate— Grade Point Average, Attendance Rate, and High School Credits. 



April 2011 



34 | P a g e 




SUMMARY OF FINDINGS 



The Findings section of this report details the differences between Academic Priority Ninth Grade Counts participants 
and Academic Priority students who didn't participate in the program. The main findings can be summarized in the three 
major categories below. 

1. Some positive effects. There was a clear positive effect on credit attainment for Academic Priority students who 
had participated in a Ninth Grade Counts program. AP participants averaged 5.59 credits at the end of ninth 
grade compared to 5.03 credits for AP nonparticipants. 56% of AP participants earned six credits or more 
compared to 49% of AP nonparticipants. When any potential credits earned in the summer program were 
subtracted, participants still earned significantly more credits— an average of 5.32 compared to 5.03 for 
nonparticipants. Participants also earned significantly more Language Arts and Science credits than 
nonparticipants. 



2. Some unclear effects. Other effects were mixed. Average attendance rates were higher for NGC AP participants 
than for AP nonparticipants after the summer program, but not significantly; both groups had lower attendance 
in ninth grade than in eighth grade. AP participants had lower average GPAs than AP nonparticipants, were less 
likely to have an A or F level GPA than nonparticipants, and were more likely to have a B, C, or D level GPA, but 
none of these differences were significant. AP participants were statistically more likely to have an expulsion or 
suspension in the ninth grade than AP nonparticipants, but data limitations make this comparison less 
meaningful. 



3. Individual program effects. Ninth Grade Counts is not a single program for which impact can be measured 
directly. It is a coordination of more than twenty individual programs in Multnomah County which all target the 
same grade level. This report seeks to measure the overall effect of NGC, but student outcomes vary based on 
which individual program they attended. Very few programs had enough Academic Priority students to measure 
individual program differences between Ninth Grade Counts participants and nonparticipants, however the two 
large programs that were able to measured showed increased outcomes for participants. Measuring the impact 
of individual programs might lead to different results than measuring Ninth Grade Counts overall. 

In summary, NWEA found that Ninth Grade Counts has a clear, positive effect on high school credit attainment. 
Academic Priority students who have participated in a program have, on average, accumulated more high school credits 
by the end of ninth grade than nonparticipants, even when adjusted for possible credits received during the summer 
and even when controlling for potential selection bias. Academic Priority Ninth Grade Counts participants don't have 
significantly higher attendance rates or higher GPAs in ninth grade compared to AP nonparticipants, and do have higher 
levels of suspensions. Preliminary analysis suggests that some individual programs could show better outcomes for 
students, but more data is needed to analyze further. In addition, evaluating the impact on dropout and graduation 
statistics would require three more years of data to be collected for this cohort. 
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This evaluation is limited by the data available, and some of the outcome variables were insufficient for evaluating 
program effect. Although reading and math test scores were collected both before and after the summer program, an 
insufficient sample of students took the test in the ninth grade, so test scores and proficiency rates could not be used for 
evaluation. Grade point average and high school credit accumulation were collected for students before and after the 
program, but most of these measures are not collected consistently for all schools, particularly in the eighth grade. Out- 
of-school suspensions and expulsions were infrequent in nature, making the comparison between disciplinary rates 
unclear. A better metric might be the number of referrals for disciplinary action, including in-school suspension. 

RECOMMENDATIONS FOR DATA COLLECTION 

In the process of performing this analysis of the Ninth Grade Counts summer 2009 program, NWEA researchers noticed 
some issues that could be addressed to increase the effectiveness of future reports for this population. Those 
recommendations are included below as the last section of this report. 

• Ensure all schools report grade point average statistics for ninth grade 

Some schools did not report grade point averages for ninth grade students. Because these students were coded with a 
GPA of 0.00, all students with a 0.00 GPA had to be removed from this analysis. Encouraging schools to report ninth 
grade GPA in the student data management system, and coding missing data as missing instead of 0, would allow for a 
more accurate measurement of this variable. 

• Measure which students received high school credit for summer program participation 

One of the main findings is that by the end of ninth grade, NGC participants accumulate a larger number of high school 
credits than nonparticipants. When it was estimated that all students received the full available credit, NGC participants 
still averaged more credits. If the number and type of credits received by each student were recorded by the program, a 
more precise reflection of the program's impact on credits could be reported. 

• Collect the names of courses students have taken, along with grades 

One of the goals of Ninth Grade Counts is to decrease the number of failing grades in core academic areas. This was 
measured using overall Grade Point Average as well as number of credits attempted and earned in core subject areas. 
Without course names, however, it is impossible to determine what kinds of classes the students are taking. For 
instance, if a ninth grade student is taking Geometry and earning a B, that may be a better outcome than earning an A in 
a basic math course. A previous pilot study of this population in January 2010 indicated that students were taking more 
arts and physical education classes than history or foreign language classes, and that grades in those classes were 
higher. The impact of Ninth Grade Counts programs on the choice of ninth grade classes could be confirmed if course 
names were available. 

• Perform a multi-year study involving a full high school cohort 

This report shows whether students in Ninth Grade Counts programs had better outcomes in the year after the summer 
program. The high-level goal of the program, however, is to re-engage students in the educational system and reduce 
their chance of dropping out before graduation. This goal can only be measured by tracking the population through an 
entire high school career— from eighth grade before the program until the year after twelfth grade. NWEA recommends 
that as part of the Portland Schools Foundation's portfolio of high school programs, it commissions a full evaluation of at 
least one cohort of students as they progress through grades eight to twelve. 
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APPENDIX A: Complete Counts of Cohort 1 Students 



Ninth Grade Counts Participants 



Academic Priority Students 



School 

District 



Race/ 

Ethnicity 



Gender 



Free/ 

Reduced 

Lunch 

Status 

English 

Language 

Learner 

Status 





All 


AP 


Not AP 




All 


NGC 


Not NGC 


[blank] 


0 


0 


0 




3 


0 


3 


Centennial School District 
28J 


13 


0 


13 




316 


0 


316 


David Douglas School 
District 40 


68 


42 


26 




374 


42 


332 


Gresham-Barlow School 
District 10J 


47 


0 


47 




383 


0 


383 


Parkrose School Di stri ct 3 


114 


60 


54 




146 


60 


86 


Portland School District 1J 


397 


221 


176 




1203 


221 


982 


Reynolds School District 7 


79 


76 


3 




441 


76 


365 


Total 


718 


399 


319 




2866 


399 


2467 


American Indian / Alaskan 
Native 


25 


18 


7 




68 


18 


50 


Asian / Pacific Islander 


70 


27 


43 




202 


27 


175 


Black / African American 


205 


113 


92 




462 


113 


349 


Hispanic 


187 


104 


83 




672 


104 


568 


White 


230 


137 


93 




1458 


137 


1321 


Other 


1 


0 


1 




4 


0 


4 


Total 


718 


399 


319 




2866 


399 


2467 


F 


343 


176 


167 




1257 


176 


1081 


M 


375 


223 


152 




1609 


223 


1386 


Total 


718 


399 


319 




2866 


399 


2467 


N 


125 


62 


63 




929 


62 


867 


Y 


593 


337 


256 




1937 


337 


1600 


Total 


718 


399 


319 




2866 


399 


2467 


N 


538 


300 


238 




2200 


300 


1900 


Y 


180 


99 


81 




666 


99 


567 


Total 


718 


399 


319 




2866 


399 


2467 
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APPENDIX B: Average Performance of Cohort 1 Students 



Academic Priority 
Nonparticipant 



Academic Priority 
Participant 



Mean N StdDevH Mean 



Non-Academic 
Priority Participant 



Mean N StdDevH Mean N Std Devi 



8th grade: 
2008-2009 



Attendance rate 


89.4% 


2460 


0.09 


Days absent 


17.12 


2464 


13.64 


Percent 90% or above 
attendance 


55% 


2460 


0.50 


Percent 85% or above 
attendance 


77% 


2460 


0.42 


Percent 80% or above 
attendance 


88% 


2460 


0.33 


Expulsion rate 


0.01 


2464 


0.08 


Suspension rate 


0.42 


2464 


1.08 


Percent no suspension 


79% 


2464 


0.41 


Math test score 


230.5 


2382 


10.35 


Percent meeting/exceeding 
on Math test 


51% 


2382 


0.50 


Reading test score 


230.1 


2377 


9.17 


Percent meeting/exceeding 
on Rea ding test 


49% 


2377 


0.50 



9th grade: 
2009-2010 



Attendance rate 


85.6% 


2308 


0.14 


Days absent 


22.12 


2308 


20.19 


Percent 90% or above 
attendance 


50% 


2308 


0.50 


Percent 85% or above 
attendance 


65% 


2308 


0.48 


Percent 80% or above 
attendance 


75% 


2308 


0.43 


Expulsion rate 


0.01 


2313 


0.11 


Suspension rate 


0.34 


2313 


0.93 


Percent no suspension 


82% 


2313 


0.39 


Grade Point Average (GPA) 


1.96 


2039 


0.98 


Percent 1.50 or above GPA 


66% 


2039 


0.47 


Percent 2.00 or above GPA 


49% 


2039 


0.50 


Percent 2.50 or above GPA 


32% 


2039 


0.47 


High school credits 
attained 


5.03 


2313 


2.63 


Percent 6 or more credits 


49% 


2313 


0.50 


Credits subtracting 
possible summer credits 


5.03 


2313 


2.63 


Percent 6 or more credits 
subtracting possible 
summer credits 


49% 


2313 


0.50 


Math credits earned 


0.69 


2313 


0.65 


Math credits attempted 


0.99 


2313 


0.58 


Language Arts credits 
earned 


0.83 


2313 


0.67 


Language Arts credits 
attempted 


1.14 


2313 


0.64 


Science credits earned 


0.59 


2313 


0.46 


Science credits attempted 


0.87 


2313 


0.37 


Social Studies credits 
earned 


0.33 


2313 


0.44 


Social Studies credits 
attempted 


0.48 


2313 


0.50 



91.0% 


398 


0.08 


15.06 


399 


14.34 


64% 


398 


0.48 


83% 


398 


0.38 


90% 


398 


0.29 


0.00 


399 


0.00 


0.45 


399 


1.05 


77% 


399 


0.42 


227.4 


392 


8.65 


40% 


392 


0.49 


227.3 


390 


8.25 


38% 


390 


0.49 



86.9% 


392 


0.11 


21.18 


392 


17.90 


49% 


392 


0.50 


66% 


392 


0.47 


79% 


392 


0.41 


0.01 


392 


0.11 


0.54 


392 


1.16 


72% 


392 


0.45 


1.89 


365 


0.89 


65% 


365 


0.48 


47% 


365 


0.50 


29% 


365 


0.46 


5.59 


392 


2.54 


56% 


392 


0.50 


5.32 


392 


2.52 


51% 


392 


0.50 


0.68 


392 


0.60 


0.98 


392 


0.53 


0.94 


392 


0.70 


1.30 


392 


0.67 


0.65 


392 


0.49 


0.97 


392 


0.41 


0.32 


392 


0.46 


0.48 


392 


0.55 



94.9% 


306 


0.07 


7.96 


306 


8.00 


90% 


306 


0.31 


95% 


306 


0.21 


97% 


306 


0.16 


0.01 


307 


0.13 


0.33 


307 


0.91 


84% 


307 


0.37 


232.7 


301 


8.87 


64% 


301 


0.48 


231.3 


297 


7.09 


59% 


297 


0.49 



92.6% 


311 


0.10 


12.22 


311 


15.47 


77% 


311 


0.42 


87% 


311 


0.33 


92% 


311 


0.27 


0.01 


311 


0.08 


0.24 


311 


0.81 


86% 


311 


0.34 


2.53 


295 


0.98 


83% 


295 


0.38 


71% 


295 


0.46 


58% 


295 


0.49 


6.82 


311 


2.55 


73% 


311 


0.44 


6.55 


311 


2.52 


70% 


311 


0.46 


1.12 


311 


0.68 


1.37 


311 


0.60 


1.15 


311 


0.65 


1.28 


311 


0.60 


0.88 


311 


0.49 


1.06 


311 


0.40 


0.39 


311 


0.53 


0.43 


311 


0.54 



90.1% 


3164 


0.09 


15.98 


3169 


13.57 


59% 


3164 


0.49 


80% 


3164 


0.40 


89% 


3164 


0.31 


0.01 


3170 


0.08 


0.42 


3170 


1.06 


79% 


3170 


0.41 


230.4 


3075 


10.09 


51% 


3075 


0.50 


229.8 


3064 


8.94 


49% 


3064 


0.50 



86.5% 


3011 


0.14 


20.97 


3011 


19.69 


53% 


3011 


0.50 


68% 


3011 


0.47 


78% 


3011 


0.42 


0.01 


3016 


0.11 


0.36 


3016 


0.95 


81% 


3016 


0.39 


2.01 


2699 


0.99 


68% 


2699 


0.47 


51% 


2699 


0.50 


35% 


2699 


0.48 


5.29 


3016 


2.66 


52% 


3016 


0.50 


5.23 


3016 


2.64 


51% 


3016 


0.50 


0.74 


3016 


0.66 


1.03 


3016 


0.59 


0.88 


3016 


0.68 


1.17 


3016 


0.64 


0.62 


3016 


0.47 


0.90 


3016 


0.38 


0.34 


3016 


0.45 


0.47 


3016 


0.51 
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APPENDIX C: District Data File Layout 



Field Name 


Field Description 


Changes from original submission 


COHORT 


Summer 2009 or Summer 2010 




STUDENTTYPE 


Participant or Academic Priority Non-Participant 




PSF_PROGRAM_ID 


Ninth Grade Counts program id 




MESD_Proxy_ID 


Number assigned by MESD staff 




eSIS_Year 


year of data (2008-2009 or 2009-2010) 




Ethnicity 


Hispanic or Not Hispanic 


New field race/ethnicity created. If ethnicity is 
Hispanic, then report Hispanic, otherwise report race. 


Race 


Any of the five individual items, or "Multi" 


Gender 


M/F 




Grade 


Student's grade level forgiven school year 




Grade_9_Entry_Date 


If applicable 




Free_Reduced_Meals 


Y/N 


If Y in either year, then Y for both years 


DistrictJD 


Currently Enrolled District ID 




District_Name 


Currently Enrolled District 




SchoolJD 


Currently Enrolled School ID 




School_Name 


Currently Enrolled School 




GPA 


Current Cumulative GPA 




ELL 


Y/N/M (M=Monitor) 


If Yor M in eitheryear, then Y for both years 


Academic_Priority 


Y/N (Y if active program "Academic Priority") 


If Y in eitheryear, then Y for both years 


Expulsions 


Count of Expulsions 




Suspensions 


Count of Suspensions - Out of School 




Days_Present 


Pform Year Total Days Present 




Days_Absent 


Pform Year Total Days Absent 




OAKS_Math_Assess_Date 


test date 




OAKS_Math_Test_Level 


test level 




OAKS_Math_Perf_Level 


performance level 




OAKS_Math_RIT 


RIT scaled test score 




OAKS_Math_Qui nti le 


performance quintile 




OAKS_Math_Pe rce nti 1 e 


performance percentile 




OAKS_Readi ng_Assess_Date 


test date 




OAKS_Readi ng_T e st_Le ve 1 


test level 




OAKS_Reading_Perf_Level 


performance level 




OAKS_Reading_RIT 


RIT scaled test score 




OAKS_Reading_Quintile 


performance quintile 




OAKS_Readi ng_Pe rce nti 1 e 


performance percentile 




HS_Credits_Earned 


Total HS diploma credits student earned to date 




C u m_M at h_Cre d i ts_Ea rn e d 


Sum of credits earned for Math courses 




Cum_Math_Credits_Attempted 


Sum of credits attempted for Math courses 




Cum_LA_Credits_Earned 


Sum of credits earned for Language Arts courses 




C u m_LA_C re d i ts_Atte m pte d 


Sum of credits attempted for Language Arts 




Cum_SC_Credits_Earned 


Sum of credits earned for Social Studies courses 




C u m_SC_C re d i ts_Atte m pte d 


Sum of credits attempted for Social Studies 




Cum_SS_Credits_Earned 


Sum of credits earned for Science courses 




C u m_SS_C re d i ts_Att e m pte d 


Sum of credits attempted for Science courses 




Comment 


explanatory comments for data 
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APPENDIX D: Example of Program Report 



Ninth Grade Counts: indixidual program name 
Summer 2009 and Summer 2010 



Demographics for Summer 2009 and Summer 2010 







Cohort l:Summer2X9 


Cohort 2: 


Summer 2010 


Summer 2010: 100 students 


#<* 

Students 


Hof 

Students 


NGC % of 
Students 


6cSstricts% 
of Students 


#af 

Students 


Sof 

Students 


NGC % of 
Students 


6districts56 
of Students 


Academic Prioritv 


















AP 


5* 


536 


536 


-3-: 


62 


62* 


48* 


42* 


Not AP 


46 




*46 


536 


38 


3 8k 


52* 


58* 


Re:e Etinni city 


















Native American 


7 


7 * 


3k 


lk 


8 


36 


26 


S6 


Asian 


13 


136 


iok 


106 


12 


12* 


10* 


9* 


African American 


28 


236 


23 k 


It: 


26 


26* 


26* 


10* 


Hispanic 


17 


1756 


206 


196 


23 


23* 


28* 


21* 


White 


29 


29* 


32* 


55: 


22 




28* 


54* 


Other 


6 


6* 


<26 


46 


9 


9* 


6* 


5* 


Gender 


















M 


4 6 


406 


526 


50: 


58 


58* 


51* 


SC* 


F 


54 


5*: 


406 


50: 


A2 


42* 


49* 


50* 


Free prReduced Lunch 


















FRL 


72 


726 


536 


5254 


79 


79* 


79* 


54* 


Not FRL 


28 


236 


17k 


406 


21 


21* 


21* 


46* 


English Language Learner 


















ELL 


26 


206 


25k 


126 


32 


32* 


24* 


12* 


Not ELL 


74 


7r*e 


736 


SO: 


63 


68* 


76* 


88* 



— Counts of stude'ts ess than £ are removed far co'fzz'l a ty 



* Cats fo' fa £ districts f-omt'e O'agt' Department of Ec.cato' weoste. taoe* ' oa'e't'e-a-. 

Race/ethnicity %fo' 5' g'ace's each year (#67|; Se'ce-%f'om -at' aaiajj-'a'tsfo' S' g-ace-a 2CCS-20l0orty (#95|; frl %fo' a 
g'aras co^o'erfo'eac yea' (#611; ELLWfrom math 3j=e”” l e't-fo' S' g'ace LEP students 2COS-2C10 ony (#95 



Outcome Measures for Summer 2009 Cohort (years before and after Summer program) 



:c»:ccs 




:cce-:o:o 


Measure 


My 

prograen 


NGC 


MyAP 


NGC AP 




Measure 


My 

pro ram 


NGC 


MyAP 


NGC AP 


At!« TdXCS itS 


923ft 


S2-7H 


39 c-t 


9* Cfi 




Atx -oa-cfi ate 


a: a* 


39 4 i i 


39 9*4 


Se 9^i 


ftofssudfi“«%y 


65* 


73ft 


44ft 


54ft 




ft o 4 st -da rs w ti 
arjndanos >9Cf = 


53ft 


62ft 


67ft 


49ft 




0 47 


040 


022 


0 45 






012 


0 41 


Oil 


094 


ftotsfludsncsv* t-ojt ; 
;-s:e *s cr. o-ex: js on 


S2ft 


soft 


S9ft 


77ft 




; i o? students w eft out a 
•; onoraxs js a- 


88* 


7«ft 


89ft 


72ft 


Vatn sco« 


232 


230 


229 


227 




GPA 


2 23 


127 


1-97 


1S5 


fto^ssjoe-tt moot ng 
•:* sta*da*d 


47ft 


31ft 


33ft 


40ft 




ft o' st.de “<ts w tn a 
liOw'iaata- 


n* 


73ft 


44ft 


53ft 


s cac -igsco-a 


232 


229 


229 


227 




- gft Scnoo Cf«o t; 


7 10 


5 3 


523 


5 39 


iio 4 s^Si*« -naa: ■» 
5 aac 'irrera 


soft 


47ft 


33ft 


3£ft 




ft o 4 studs -r.i v* tn S O' 
■x e rad ts 


65* 


54ft 


44ft 


35ft 



— Counts of st.ce-ts ass fa- £ a-a removed fen co-f dent a rty 
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